Assessing the Potential of Classical Q-learning in General Game Playing

After the recent groundbreaking results of AlphaGo and AlphaZero, we have seen strong interests in deep reinforcement learning and artificial general intelligence (AGI) in game playing. However, deep learning is resource-intensive and the theory is not yet well developed. For small games, simple classical table-based Q-learning might still be the algorithm of choice. General Game Playing (GGP) provides a good testbed for reinforcement learning to research AGI. Q-learning is one of the canonical reinforcement learning methods, and has been used by (Banerjee $\&$ Stone, IJCAI 2007) in GGP. In this paper we implement Q-learning in GGP for three small-board games (Tic-Tac-Toe, Connect Four, Hex)\footnote{source code: https://github.com/wh1992v/ggp-rl}, to allow comparison to Banerjee et al.. We find that Q-learning converges to a high win rate in GGP. For the $\epsilon$-greedy strategy, we propose a first enhancement, the dynamic $\epsilon$ algorithm. In addition, inspired by (Gelly $\&$ Silver, ICML 2007) we combine online search (Monte Carlo Search) to enhance offline learning, and propose QM-learning for GGP. Both enhancements improve the performance of classical Q-learning. In this work, GGP allows us to show, if augmented by appropriate enhancements, that classical table-based Q-learning can perform well in small games.Comment: arXiv admin note: substantial text overlap with arXiv:1802.0594

In silico Assessment of Drug-like Properties of Alkaloids from Areca catechu L Nut

Purpose: To investigate in silico the drug-like properties of alkaloids (arecoline, arecaidine, guvacine, guvacoline, isoguvacine, arecolidine and homoarecoline) obtained from the fruits of Areca catechu L (areca nut).Methods: All chemical structures were re-drawn using Chemdraw Ultra 11.0. Furthermore, software including Bio-Loom for Windows - version 1.5, Molinspiration Property Calculator and ACD/I-LAB service were used to predict the drug-like properties of the alkaloids, including relative molecular mass (MW), partition coefficient log P (cLog P), number of hydrogen bond donors (HBD), number of hydrogen bond acceptors (HBA), topological polar surface area (TPSA), number of rotatable bonds (NROTB), pKa, and aqueous solubility at a given pH (LogS). In addition, Lipinski’s rule was used to evaluate druglike properties.Results: From our research, MWs of the seven compounds were all < 500. HBD and cLog P values of the seven compounds were all < 5, and HBA values were all < 10. In addition, TPSA value of each compound was < 60 Å2, and NROTB value was < 10. Besides, pKa values of the seven alkaloids were > 7.5; furthermore, they possess good solubility at pH 1.0, 5.0, and 7.0.Conclusion: All the seven alkaloids possess good drug-like properties, and demonstrated good oral absorption and bioavailability. The results also suggest that these compounds can be further developed into new oral drugs for treating certain diseases.Keywords: Areca catechu L, Areca nut, Drug-like properties, Alkaloids, Arecoline, Arecaidine, Guvacine, Guvacoline, Isoguvacine, Arecolidine, Homoarecoline, In silic

Jugular venous reflux and white matter abnormalities in Alzheimer's disease: a pilot study.

To determine whether jugular venous reflux (JVR) is associated with cerebral white matter changes (WMCs) in individuals with Alzheimer's disease (AD), we studied 12 AD patients 24 mild cognitive impairment (MCI) patients, and 17 elderly age- and gender-matched controls. Duplex ultrasonography and 1.5T MRI scanning was applied to quantify cerebral WMCs [T2 white matter (WM) lesion and dirty-appearing-white-matter (DAWM)]. Subjects with severe JVR had more frequently hypertension (p = 0.044), more severe WMC, including increased total (p = 0.047) and periventricular DAWM volumes (p = 0.008), and a trend for increased cerebrospinal fluid volumes (p = 0.067) compared with the other groups. A significantly decreased (65.8%) periventricular DAWM volume (p = 0.01) in the JVR-positive AD individuals compared with their JVR-negative counterparts was detected. There was a trend for increased periventricular and subcortical T2 WMC lesion volumes in the JVR-positive AD individuals compared with their JVR-negative counterparts (p = 0.073). This phenomenon was not observed in either the control or MCI groups. In multiple regression analysis, the increased periventricular WMC lesion volume and decreased DAWM volume resulted in 85.7% sensitivity and 80% specificity for distinguishing between JVR-positive and JVR-negative AD patients. These JVR-WMC association patterns were not seen in the control and MCI groups. Therefore, this pilot study suggests that there may be an association between JVR and WMCs in AD patients, implying that cerebral venous outflow impairment might play a role in the dynamics of WMCs formation in AD patients, particularly in the periventricular regions. Further longitudinal studies are needed to confirm and validate our findings

Molecular differences between ductal carcinoma in situ and adjacent invasive breast carcinoma: a multiplex ligation-dependent probe amplification study

Ductal carcinoma in situ (DCIS) accounts for approximately 20% of mammographically detected breast cancers. Although DCIS is generally highly curable, some women with DCIS will develop life-threatening invasive breast cancer, but the determinants of progression to infiltrating ductal cancer (IDC) are largely unknown. In the current study, we used multiplex ligation-dependent probe amplification (MLPA), a multiplex PCR-based test, to compare copy numbers of 21 breast cancer related genes between laser-microdissected DCIS and adjacent IDC lesions in 39 patients. Genes included in this study were ESR1, EGFR, FGFR1, ADAM9, IKBKB, PRDM14, MTDH, MYC, CCND1, EMSY, CDH1, TRAF4, CPD, MED1, HER2, CDC6, TOP2A, MAPT, BIRC5, CCNE1 and AURKA

First insights into the phylogenetic diversity of Mycobacterium tuberculosis in Nepal

BACKGROUND: Tuberculosis (TB) is a major public health problem in Nepal. Strain variation in Mycobacterium tuberculosis may influence the outcome of TB infection and disease. To date, the phylogenetic diversity of M. tuberculosis in Nepal is unknown. METHODS AND FINDINGS: We analyzed 261 M. tuberculosis isolates recovered from pulmonary TB patients recruited between August 2009 and August 2010 in Nepal. M. tuberculosis lineages were determined by single nucleotide polymorphisms (SNP) typing and spoligotyping. Drug resistance was determined by sequencing the hot spot regions of the relevant target genes. Overall, 164 (62.8%) TB patients were new, and 97 (37.2%) were previously treated. Any drug resistance was detected in 50 (19.2%) isolates, and 16 (6.1%) were multidrug-resistant. The most frequent M. tuberculosis lineage was Lineage 3 (CAS/Delhi) with 106 isolates (40.6%), followed by Lineage 2 (East-Asian lineage, includes Beijing genotype) with 84 isolates (32.2%), Lineage 4 (Euro-American lineage) with 41 (15.7%) isolates, and Lineage 1 (Indo-Oceanic lineage) with 30 isolates (11.5%). Based on spoligotyping, we found 45 different spoligotyping patterns that were previously described. The Beijing (83 isolates, 31.8%) and CAS spoligotype (52, 19.9%) were the dominant spoligotypes. A total of 36 (13.8%) isolates could not be assigned to any known spoligotyping pattern. Lineage 2 was associated with female sex (adjusted odds ratio [aOR] 2.58, 95% confidence interval [95% CI] 1.42-4.67, p = 0.002), and any drug resistance (aOR 2.79; 95% CI 1.43-5.45; p = 0.002). We found no evidence for an association of Lineage 2 with age or BCG vaccination status. CONCLUSIONS: We found a large genetic diversity of M. tuberculosis in Nepal with representation of all four major lineages. Lineages 3 and 2 were dominating. Lineage 2 was associated with clinical characteristics. This study fills an important gap on the map of the M. tuberculosis genetic diversity in the Asian reg

Occupational and leisure time physical activity in contrasting relation to ambulatory blood pressure

Background: While moderate and vigorous leisure time physical activities are well documented to decrease the risk for cardiovascular disease, several studies have demonstrated an increased risk for cardiovascular disease in workers with high occupational activity. Research on the underlying causes to the contrasting effects of occupational and leisure time physical activity on cardiovascular health is lacking. The aim of this study was to examine the relation of objective and self-report measures of occupational and leisure time physical activity with 24-h ambulatory systolic blood pressure (BP). Methods: Results for self-reported physical activity are based on observations in 182 workers (60% male, mean age 51 years), while valid objective physical activity data were available in 151 participants. The usual level of physical activity was assessed by 5 items from the Job Content Questionnaire (high physical effort, lifting heavy loads, rapid physical activity, awkward body positions and awkward positions of head or arms at work) and one item asking about the general level of physical activity during non-working time. On a regular working day, participants wore an ambulatory BP monitor and an accelerometer physical activity monitor during 24 h. Associations were examined by means of Analysis of Covariance. Results: Workers with an overall high level of self-reported occupational physical activity as well as those who reported to often lift heavy loads at work had a higher mean systolic BP at work, at home and during sleep. However, no associations were observed between objectively measured occupational physical activity and BP. In contrast, those with objectively measured high proportion of moderate and vigorous leisure time physical activity had a significantly lower mean systolic BP during daytime, while no differences were observed according to self-reported level of leisure time physical activity. Conclusions: These findings suggest that workers reporting static occupational physical activities, unlike general physically demanding tasks characterized by dynamic movements of large muscle groups, are related to a higher daily systolic BP, while high objective levels of moderate and vigorous leisure time physical activity are related to lower daytime systolic BP. Ambulatory systolic BP may be a physiological explanatory factor for the contrasting effects of occupational and leisure time physical activity

Seroprevalence of avian influenza A (H5N1) virus among poultry workers in Jiangsu Province, China: an observational study

<p>Abstract</p> <p>Background</p> <p>Since 2003 to 06 Jan 2012, the number of laboratory confirmed human cases of infection with avian influenza in China was 41 and 27 were fatal. However, the official estimate of the H5N1 case-fatality rate has been described by some as an over estimation since there may be numerous undetected asymptomatic/mild cases of H5N1 infection. This study was conducted to better understand the real infection rate and evaluate the potential risk factors for the zoonotic spread of H5N1 viruses to humans.</p> <p>Methods</p> <p>A seroepidemiological survey was conducted in poultry workers, a group expected to have the highest level of exposure to H5N1-infected birds, from 3 counties with habitat lakes of wildfowl in Jiangsu province, China. Serum specimens were collected from 306 participants for H5N1 serological test. All participants were interviewed to collect information about poultry exposures.</p> <p>Results</p> <p>The overall seropositive rate was 2.61% for H5N1 antibodies. The poultry number was found associated with a 2.39-fold significantly increased subclinical infection risk after adjusted with age and gender.</p> <p>Conclusions</p> <p>Avian-to -human transmission of avian H5N1 virus remained low. Workers associated with raising larger poultry flocks have a higher risk on seroconversion.</p

Modification of EGF-Like Module 1 of Thrombospondin-1, an Animal Extracellular Protein, by O-Linked N-Acetylglucosamine

Thrombospondin-1 (TSP-1) is known to be subject to three unusual carbohydrate modifications: C-mannosylation, O-fucosylation, and O-glucosylation. We now describe a fourth: O-β-N-acetylglucosaminylation. Previously, O-β-N-acetylglucosamine (O-β-GlcNAc) was found on a threonine in the loop between the fifth and sixth cysteines of the 20th epidermal growth factor (EGF)-like module of Drosophila Notch. A BLAST search based on the Drosophila Notch loop sequence identified a number of human EGF-like modules that contain a similar sequence, including EGF-like module 1 of TSP-1 and its homolog, TSP-2. TSP-1, which has a potentially modifiable serine in the loop, reacted in immuno-blots with the CTD110.6 anti-O-GlcNAc antibody. Antibody reactivity was diminished by treatment of TSP-1 with β-N-acetylhexosaminidase. TSP-2, which lacks a potentially modifiable serine/threonine in the loop, did not react with CTD110.6. Analysis of tandem modules of TSP-1 localized reactivity of CTD110.6 to EGF-like module 1. Top-down mass spectrometric analysis of EGF-like module 1 demonstrated the expected modifications with glucose (+162 Da) and xylose (+132 Da) separately from modification with N-acetyl hexosamine (+203 Da). Mass spectrometric sequence analysis localized the +203-Da modification to Ser580 in the sequence 575CPPGYSGNGIQC586. These results demonstrate that O-β-N-acetylglucosaminylation can occur on secreted extracellular matrix proteins as well as on cell surface proteins

The Cyprinodon variegatus genome reveals gene expression changes underlying differences in skull morphology among closely related species

Genes in durophage intersection set at 15 dpf. This is a comma separated table of the genes in the 15 dpf durophage intersection set. Given are edgeR results for each pairwise comparison. Columns indicating whether a gene is included in the intersection set at a threshold of 1.5 or 2 fold are provided. (CSV 13 kb